Multisensory training improves the development of spatial cognition after sight restoration from congenital cataracts.

Spatial cognition and mobility are typically impaired in congenitally blind individuals, as vision usually calibrates space perception by providing the most accurate distal spatial cues. We have previously shown that sight restoration from congenital bilateral cataracts guides the development of more accurate space perception, even when cataract removal occurs years after birth. However, late cataract-treated individuals do not usually reach the performance levels of the typically sighted population. Here, we developed a brief multisensory training that associated audiovisual feedback with body movements. Late cataract-treated participants quickly improved their space representation and mobility, performing as well as typically sighted controls in most tasks. Their improvement was comparable with that of a group of blind participants, who underwent training coupling their movements with auditory feedback alone. These findings suggest that spatial cognition can be enhanced by a training program that strengthens the association between bodily movements and their sensory feedback (either auditory or audiovisual).

Visual experience shapes the Bouba-Kiki effect and the size-weight illusion upon sight restoration from congenital blindness.

The Bouba-Kiki effect is the systematic mapping between round/spiky shapes and speech sounds ("Bouba"/"Kiki"). In the size-weight illusion, participants judge the smaller of two equally-weighted objects as being heavier. Here we investigated the contribution of visual experience to the development of these phenomena. We compared three groups: early blind individuals (no visual experience), individuals treated for congenital cataracts years after birth (late visual experience), and typically sighted controls (visual experience from birth). We found that, in cataract-treated participants (tested visually/visuo-haptically), both phenomena are absent shortly after sight onset, just like in blind individuals (tested haptically). However, they emerge within months following surgery, becoming statistically indistinguishable from the sighted controls. This suggests a pivotal role of visual experience and refutes the existence of an early sensitive period: A short period of experience, even when gained only years after birth, is sufficient for participants to visually pick-up regularities in the environment, contributing to the development of these phenomena.

Grasping behavior does not recover after sight restoration from congenital blindness.

We investigated whether early visual input is essential for establishing the ability to use predictions in the control of actions and for perception. To successfully interact with objects, it is necessary to pre-program bodily actions such as grasping movements (feedforward control). Feedforward control requires a model for making predictions, which is typically shaped by previous sensory experience and interaction with the environment.1 Vision is the most crucial sense for establishing such predictions.2,3 We typically rely on visual estimations of the to-be-grasped object's size and weight in order to scale grip force and hand aperture accordingly.4,5,6 Size-weight expectations play a role also for perception, as evident in the size-weight illusion (SWI), in which the smaller of two equal-weight objects is misjudged to be heavier.7,8 Here, we investigated predictions for action and perception by testing the development of feedforward controlled grasping and of the SWI in young individuals surgically treated for congenital cataracts several years after birth. Surprisingly, what typically developing individuals do easily within the first years of life, namely to adeptly grasp new objects based on visually predicted properties, cataract-treated individuals did not learn after years of visual experience. Contrary, the SWI exhibited significant development. Even though the two tasks differ in substantial ways, these results may suggest a potential dissociation in using visual experience to make predictions about an object's features for perception or action. What seems a very simple task-picking up small objects-is in truth a highly complex computation that necessitates early structured visual input to develop.

Development of local-global preference in vision and haptics.

We aimed to advance our understanding of local-global preference by exploring its developmental path within and across sensory modalities: vision and haptics. Neurotypical individuals from six years of age through adulthood completed a similarity judgement task with hierarchical haptic or visual stimuli made of local elements (squares or triangles) forming a global shape (a square or a triangle). Participants chose which of two probes was more similar to a target: the one sharing the global shape (but different local shapes) or the one with the same local shapes (but different global shape). Across trials, we independently varied the size of the local elements and that of the global configuration-the latter was varied by manipulating local element density while keeping their numerosity constant. We found that the size of local elements (but not global size) modulates the effects of age and modality. For stimuli with smaller local elements, the proportion of global responses increased with age and was similar for visual and haptic stimuli. However, for stimuli made of our largest local elements, the global preference was reduced or absent, particularly in haptics, regardless of age. These results suggest that vision and haptics progressively converge toward similar global preference with age, but residual differences across modalities and across individuals may be observed, depending on the characteristics of the stimuli.

Crossmodal plasticity following short-term monocular deprivation.

A brief period of monocular deprivation (MD) induces short-term plasticity of the adult visual system. Whether MD elicits neural changes beyond visual processing is yet unclear. Here, we assessed the specific impact of MD on neural correlates of multisensory processes. Neural oscillations associated with visual and audio-visual processing were measured for both the deprived and the non-deprived eye. Results revealed that MD changed neural activities associated with visual and multisensory processes in an eye-specific manner. Selectively for the deprived eye, alpha synchronization was reduced within the first 150 ms of visual processing. Conversely, gamma activity was enhanced in response to audio-visual events only for the non-deprived eye within 100-300 ms after stimulus onset. The analysis of gamma responses to unisensory auditory events revealed that MD elicited a crossmodal upweight for the non-deprived eye. Distributed source modeling suggested that the right parietal cortex played a major role in neural effects induced by MD. Finally, visual and audio-visual processing alterations emerged for the induced component of the neural oscillations, indicating a prominent role of feedback connectivity. Results reveal the causal impact of MD on both unisensory (visual and auditory) and multisensory (audio-visual) processes and, their frequency-specific profiles. These findings support a model in which MD increases excitability to visual events for the deprived eye and audio-visual and auditory input for the non-deprived eye.

The power of vision: calibration of auditory space after sight restoration from congenital cataracts.

Early visual deprivation typically results in spatial impairments in other sensory modalities. It has been suggested that, since vision provides the most accurate spatial information, it is used for calibrating space in the other senses. Here we investigated whether sight restoration after prolonged early onset visual impairment can lead to the development of more accurate auditory space perception. We tested participants who were surgically treated for congenital dense bilateral cataracts several years after birth. In Experiment 1 we assessed participants' ability to understand spatial relationships among sounds, by asking them to spatially bisect three consecutive, laterally separated sounds. Participants performed better after surgery than participants tested before. However, they still performed worse than sighted controls. In Experiment 2, we demonstrated that single sound localization in the two-dimensional frontal plane improves quickly after surgery, approaching performance levels of sighted controls. Such recovery seems to be mediated by visual acuity, as participants gaining higher post-surgical visual acuity performed better in both experiments. These findings provide strong support for the hypothesis that vision calibrates auditory space perception. Importantly, this also demonstrates that this process can occur even when vision is restored after years of visual deprivation.

Recalibrating vision-for-action requires years after sight restoration from congenital cataracts.

Being able to perform adept goal-directed actions requires predictive, feed-forward control, including a mapping between the visually estimated target locations and the motor commands reaching for them. When the mapping is perturbed, e.g., due to muscle fatigue or optical distortions, we are quickly able to recalibrate the sensorimotor system to update this mapping. Here, we investigated whether early visual and visuomotor experience is essential for developing sensorimotor recalibration. To this end, we assessed young individuals deprived of pattern vision due to dense congenital bilateral cataracts who were surgically treated for sight restoration only years after birth. We compared their recalibration performance to such distortion to that of age-matched sighted controls. Their sensorimotor recalibration performance was impaired right after surgery. This finding cannot be explained by their still lower visual acuity alone, since blurring vision in controls to a matching degree did not lead to comparable behavior. Nevertheless, the recalibration ability of cataract-treated participants gradually improved with time after surgery. Thus, the lack of early pattern vision affects visuomotor recalibration. However, this ability is not lost but slowly develops after sight restoration, highlighting the importance of sensorimotor experience gained late in life.

Development of multisensory integration following prolonged early-onset visual deprivation.

Adult humans make effortless use of multisensory signals and typically integrate them in an optimal fashion.1 This remarkable ability takes many years for normally sighted children to develop.2,3 Would individuals born blind or with extremely low vision still be able to develop multisensory integration later in life when surgically treated for sight restoration? Late acquisition of such capability would be a vivid example of the brain's ability to retain high levels of plasticity. We studied the development of multisensory integration in individuals suffering from congenital dense bilateral cataract, surgically treated years after birth. We assessed cataract-treated individuals' reliance on their restored visual abilities when estimating the size of an object simultaneously explored by touch. Within weeks to months after surgery, when combining information from vision and touch, they developed a multisensory weighting behavior similar to matched typically sighted controls. Next, we tested whether cataract-treated individuals benefited from integrating vision with touch by increasing the precision of size estimates, as it occurs when integrating signals in a statistically optimal fashion.1 For participants retested multiple times, such a benefit developed within months after surgery to levels of precision indistinguishable from optimal behavior. To summarize, the development of multisensory integration does not merely depend on age, but requires extensive multisensory experience with the world, rendered possible by the improved post-surgical visual acuity. We conclude that early exposure to multisensory signals is not essential for the development of multisensory integration, which can still be acquired even after many years of visual deprivation.

Aim and Plausibility of Action Chains Remap Peripersonal Space.

Successful interaction with objects in the peripersonal space requires that the information relative to current and upcoming positions of our body is continuously monitored and updated with respect to the location of target objects. Voluntary actions, for example, are known to induce an anticipatory remapping of the peri-hand space (PHS, i.e., the space near the acting hand) during the very early stages of the action chain: planning and initiating an object grasp increase the interference exerted by visual stimuli coming from the object on touches delivered to the grasping hand, thus allowing for hand-object position monitoring and guidance. Voluntarily grasping an object, though, is rarely performed in isolation. Grasping a candy, for example, is most typically followed by concatenated secondary action steps (bringing the candy to the mouth and swallowing it) that represent the agent's ultimate intention (to eat the candy). However, whether and when complex action chains remap the PHS remains unknown, just as whether remapping is conditional to goal achievability (e.g., candy-mouth fit). Here we asked these questions by assessing changes in visuo-tactile interference on the acting hand while participants had to grasp an object serving as a support for an elongated candy, and bring it toward their mouth. Depending on its orientation, the candy could potentially enter the participants' mouth (plausible goal), or not (implausible goal). We observed increased visuo-tactile interference at relatively late stages of the action chain, after the object had been grasped, and only when the action goal was plausible. These findings suggest that multisensory interactions during action execution depend upon the final aim and plausibility of complex goal-directed actions, and extend our knowledge about the role of peripersonal space in guiding goal-directed voluntary actions.

Superimposed Skilled Performance in a Virtual Mirror Improves Motor Performance and Cognitive Representation of a Full Body Motor Action.

Feedback is essential for skill acquisition as it helps identifying and correcting performance errors. Nowadays, Virtual Reality can be used as a tool to guide motor learning, and to provide innovative types of augmented feedback that exceed real world opportunities. Concurrent feedback has shown to be especially beneficial for novices. Moreover, watching skilled performances helps novices to acquire a motor skill, and this effect depends on the perspective taken by the observer. To date, however, the impact of watching one's own performance together with full body superimposition of a skilled performance, either from the front or from the side, remains to be explored. Here we used an immersive, state-of-the-art, low-latency cave automatic virtual environment (CAVE), and we asked novices to perform squat movements in front of a virtual mirror. Participants were assigned to one of three concurrent visual feedback groups: participants either watched their own avatar performing full body movements or were presented with the movement of a skilled individual superimposed on their own performance during movement execution, either from a frontal or from a side view. Motor performance and cognitive representation were measured in order to track changes in movement quality as well as motor memory across time. Consistent with our hypotheses, results showed an advantage of the groups that observed their own avatar performing the squat together with the superimposed skilled performance for some of the investigated parameters, depending on perspective. Specifically, for the deepest point of the squat, participants watching the squat from the front adapted their height, while those watching from the side adapted their backward movement. In a control experiment, we ruled out the possibility that the observed improvements were due to the mere fact of performing the squat movements-irrespective of the type of visual feedback. The present findings indicate that it can be beneficial for novices to watch themselves together with a skilled performance during execution, and that improvement depends on the perspective chosen.

Modulation frequency as a cue for auditory speed perception.

Unlike vision, the mechanisms underlying auditory motion perception are poorly understood. Here we describe an auditory motion illusion revealing a novel cue to auditory speed perception: the temporal frequency of amplitude modulation (AM-frequency), typical for rattling sounds. Naturally, corrugated objects sliding across each other generate rattling sounds whose AM-frequency tends to directly correlate with speed. We found that AM-frequency modulates auditory speed perception in a highly systematic fashion: moving sounds with higher AM-frequency are perceived as moving faster than sounds with lower AM-frequency. Even more interestingly, sounds with higher AM-frequency also induce stronger motion aftereffects. This reveals the existence of specialized neural mechanisms for auditory motion perception, which are sensitive to AM-frequency. Thus, in spatial hearing, the brain successfully capitalizes on the AM-frequency of rattling sounds to estimate the speed of moving objects. This tightly parallels previous findings in motion vision, where spatio-temporal frequency of moving displays systematically affects both speed perception and the magnitude of the motion aftereffects. Such an analogy with vision suggests that motion detection may rely on canonical computations, with similar neural mechanisms shared across the different modalities.

Infants' Visual Recognition of Pincer Grip Emerges Between 9 and 12 Months of Age.

The development of the ability to recognize the whole human body shape has long been investigated in infants, while less is known about their ability to recognize the shape of single body parts, and in particular their biomechanical constraints. This study aimed to explore whether 9- and 12-month-old infants have knowledge of a hand-grasping movement (i.e., pincer grip), being able to recognize violations of the hand's anatomical constraints during the observation of that movement. Using a preferential looking paradigm, we showed that 12-month-olds discriminate between biomechanically possible and impossible pincer grips, preferring the former over the latter (Experiment 1). This capacity begins to emerge by 9 months of age, modulated by infants' own sensorimotor experience with pincer grip (Experiment 2). Our findings indicate that the ability to visually discriminate between pincer grasps differing in their biomechanical properties develops between 9 and 12 months of age, and that experience with self-produced hand movements might help infants in building a representation of the hand that encompasses knowledge of the physical constraints of this body part.

Seeing Touches Early in Life.

The sense of touch provides fundamental information about the surrounding world, and feedback about our own actions. Although touch is very important during the earliest stages of life, to date no study has investigated infants' abilities to process visual stimuli implying touch. This study explores the developmental origins of the ability to visually recognize touching gestures involving others. Looking times and orienting responses were measured in a visual preference task, in which participants were simultaneously presented with two videos depicting a touching and a no-touching gesture involving human body parts (face, hand) and/or an object (spoon). In Experiment 1, 2-day-old newborns and 3-month-old infants viewed two videos: in one video a moving hand touched a static face, in the other the moving hand stopped before touching it. Results showed that only 3-month-olds, but not newborns, differentiated the touching from the no-touching gesture, displaying a preference for the former over the latter. To test whether newborns could manifest a preferential visual response when the touched body part is different from the face, in Experiment 2 newborns were presented with touching/no-touching gestures in which a hand or an inanimate object-i.e., a spoon- moved towards a static hand. Newborns were able to discriminate a hand-to-hand touching gesture, but they did not manifest any preference for the object-to-hand touch. The present findings speak in favour of an early ability to visually recognize touching gestures involving the interaction between human body parts.

Hearing in slow-motion: Humans underestimate the speed of moving sounds.

Perception can often be described as a statistically optimal inference process whereby noisy and incomplete sensory evidence is combined with prior knowledge about natural scene statistics. Previous evidence has shown that humans tend to underestimate the speed of unreliable moving visual stimuli. This finding has been interpreted in terms of a Bayesian prior favoring low speed, given that in natural visual scenes objects are mostly stationary or slowly-moving. Here we investigated whether an analogous tendency to underestimate speed also occurs in audition: even if the statistics of the visual environment seem to favor low speed, the statistics of the stimuli reaching the individual senses may differ across modalities, hence potentially leading to different priors. Here we observed a systematic bias for underestimating the speed of unreliable moving sounds. This finding suggests the existence of a slow-motion prior in audition, analogous to the one previously found in vision. The nervous system might encode the overall statistics of the world, rather than the specific properties of the signals reaching the individual senses.

Altered visual feedback modulates cortical excitability in a mirror-box-like paradigm.

Watching self-generated unilateral hand movements reflected in a mirror-oriented along the midsagittal plane-enhances the excitability of the primary motor cortex (M1) ipsilateral to the moving hand of the observer. Mechanisms detecting sensory-motor conflicts generated by the mirror reflection of such movements might mediate this effect; if so, cortical excitability should be modulated by the magnitude of sensory-motor conflict. To this end, we explored the modulatory effects of an altered visual feedback on M1 excitability in a mirror-box-like paradigm, by increasing or decreasing the speed of the observed movement. Healthy subjects performed movements with their left index finger while watching a video of a hand superimposed to their right static hand, which was hidden from view. The hand observed in the video executed the same movement as the observer's left hand, but at slower, same, or faster paces. Motor evoked potentials (MEPs) induced by transcranial magnetic stimulation were measured from the first dorsal interosseous and the abductor digiti minimi of the participant's hidden resting hand. The excitability of the M1 ipsilateral to the moving hand was systematically modulated by the speed of the observed hand movement: the slower the observed movement, the greater the MEP amplitude from both muscles. This evidence shows that the magnitude of the visual-motor conflicts can be used to adjust the activity of the observer's motor system. Hence, an appropriate alteration of the visual feedback, here the reduction in the movement speed, may be useful to increase its modulatory effect on motor cortical excitability.

The effect of biomechanical properties of motion on infants' perception of goal-directed grasping actions.

From a very young age, infants perceive others' actions as goal directed. Yet, the processes underlying this competence are still debated. In this study, we investigated whether (a) 4- and 6-month-old infants and adults discriminate the biomechanical properties of the human hand within an action context, (b) the manipulation of the biomechanics of hand movements has an impact on the ability to anticipate the goal of an action, and (c) the emergence of motor experience with grasping is related to infants' ability to discriminate the biomechanics of hand movements and to anticipate the action goal. The 6-month-olds discriminated between biomechanically possible and impossible grasps, and in some (but not all) instances they made more anticipatory gaze shifts toward the goal of the possible action. Both the 4- and 6-month-olds' processing of biomechanical properties of the hand were significantly related to their ability to anticipate the goal of a grasping action. Importantly, those 4-month-olds with higher precision grasping skills manifested faster anticipatory gazes toward the goal of the action. These findings suggest that multiple sources of information from an action scene are interdependent and that both perceptual information and motor experience with an action are relevant for on-line prediction of the final goal of the action.

Discrimination of biomechanically possible and impossible hand movements at birth.

The development of human body perception has long been investigated, but little is known about its early origins. This study focused on how a body part highly relevant to the human species, namely the hand, is perceived a few days after birth. Using a preferential-looking paradigm, 24- to 48-hr-old newborns watched biomechanically possible and impossible dynamic hand gestures (Experiment 1, N = 15) and static hand postures (Experiment 2, N = 15). In Experiment 1, newborns looked longer at the impossible, compared to the possible, hand movement, whereas in Experiment 2 no visual preference emerged. These findings suggest that early in life the representation of the human body may be shaped by sensory-motor experience.

The Marble-Hand Illusion.

Our body is made of flesh and bones. We know it, and in our daily lives all the senses constantly provide converging information about this simple, factual truth. But is this always the case? Here we report a surprising bodily illusion demonstrating that humans rapidly update their assumptions about the material qualities of their body, based on their recent multisensory perceptual experience. To induce a misperception of the material properties of the hand, we repeatedly gently hit participants' hand with a small hammer, while progressively replacing the natural sound of the hammer against the skin with the sound of a hammer hitting a piece of marble. After five minutes, the hand started feeling stiffer, heavier, harder, less sensitive, unnatural, and showed enhanced Galvanic skin response (GSR) to threatening stimuli. Notably, such a change in skin conductivity positively correlated with changes in perceived hand stiffness. Conversely, when hammer hits and impact sounds were temporally uncorrelated, participants did not spontaneously report any changes in the perceived properties of the hand, nor did they show any modulation in GSR. In two further experiments, we ruled out that mere audio-tactile synchrony is the causal factor triggering the illusion, further demonstrating the key role of material information conveyed by impact sounds in modulating the perceived material properties of the hand. This novel bodily illusion, the 'Marble-Hand Illusion', demonstrates that the perceived material of our body, surely the most stable attribute of our bodily self, can be quickly updated through multisensory integration.